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DESCRIPTION 

Image processing apparatus and image processing method 
BACKGROUND OF THE INVENTION 
L ToGhnical Field of the Invention 

[0001] The present invention relates to an image processing apparatus and an image 
processing method. More particularly, the present invention relates to an image 
processing apparatus and an image processing method suited for portable apparatuses, 
such as digital cameras and digital video cameras. 

2. Description of the Related Ar t Background Art 

[0002] In imaging apparatuses, such as digital cameras and digital video cameras, 
raw data (RGB data) obtained by A/D converting output signals of an imaging device is 
initially stored in a memory. Then, the raw data stored in the memory is converted 
into YC data by an image processing circuit. The image processing circuit is provided 
with a temporary storage circuit for writing the raw data read from the memory and an 
arithmetic circuit for executing arithmetic processing using the raw data of plural 
pixels read from the temporary storage circmt. Fig. 38 shows an example of a sequence 
wherein the raw data read from the memory is written to the temporary storage circuit 
for conversion processing. Specifically, the raw data is read sequentially for every 
horizontal pixel row (line) in one image 300. The conversion process requires plural 
adjacent pixels in the horizontal direction and/or vertical direction. Therefore, the 
writing sequence shown in Fig. 38 reqvdres a large-capacity Une memory capable of 
storing the pixels included in one line of the image 300 as a temporary storage circuit, 
thereby the image processing circuit becomes large in size. 
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[0003] Japanese Laid-open Patent Application No. 2000-354193 discloses that the 
image 300 is divided into plxiral blocks (image blocks 301) as shown in Fig. 39 and that 
the raw data in every image block 301 is read from the memory and processed. When 
the reading method shown in Fig. 39 is adopted, the line memory shoxild only have a 
5 storage capacity capable of storing the pixels included in one line of the image block 
301. This reduction in the storage capacity of the line memory can reduce the size of 
the image processing circuit. 

Diocloouro o f Brief Summary of the Invention 

1 0 [0004] However, one image 300 divided into image blocks 301 as shown in Fig. 39 
increases the frequency of line updating. Further, one image 300 divided into image 
blocks 301 requires updating of the image blocks 301. For example, when it is assessed 
that the number of pixels in the vertical direction in the image 300 is jA3:8 -l,218 and 
that the total number of the image blocks 301 included in the image 300 is 476, the 

1 5 number of hne updating times in the case of the processing shown in Fig. 38 is 

1317 1,217 . However, the number of line updating times in the case of the processing 
shown in Fig. 39, obtained as the product of the number of Hne updating times (65) in 
one image block 301 and the total nimiber (475) of the image blocks 301, reaches 
a08? 630.875 . 

2 0 [0005] When the reading of raw data in each image block 301 is carried out merely in 
every line as shown in Fig. 39, unconvertible combinations are formed in the raw data 
read from the memory to the temporary storage circuit at hne updating and image 
block updating. While the raw data stored in the temporary storage circuit forms an 
unconvertible combination, the arithmetic circuit can not generate effective output 

2 5 pixels, whereby data in the hne memory and registers constituting the temporary 
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storage circuit is merely shifted. 

[0006] Whon a n For example , when whoroin conversion processing is executed by 
using the pixel data of three pixels in the horizontal direction and three pixels in the 
vertical direction, conversion processing cannot be carried out until the pixels of at 
5 least the third pixel 302 from the head of a new line is written to the temporary storage 
circuit as shown in Fig. 40 at the line updating. Further, at the updating of the image 
block 301, conversion processing cannot be carried out until the pixels of at least the 
third pixel 302 from the head of the third line is written as shown in Fig. 41. In the 
case when the image 300 is divided into the image blocks 301 as described above, 
10 numerous lines and image blocks need to be updated. Therefore, time loss during the 
updating of the lines and image blocks is not negUgible, thereby reducing processing 
speed and increasing power consumption. 

[0007] Accordingly, an object of the present invention is to eliminate the processing 
loss at the updating of the lines and images, thereby improving processing speed and 

1 5 reducing power consumption. 

[0008] A first aspect of the present invention provides an image processing apparatus 
compriGing. including an image storage section adapted to store an image, an 
arithmetic section adapted to generate output pixels by arithmetically processing 
adjacent pixels consisting of M pixels adjoining in the horizontal direction and N pixels 

2 0 adjoining in the vertical direction of the image, the M being an integer of two or more, 
and the N being an integer of one or more, a first temporary storage (reading out) 
section adapted to readably store the pixels ranging from a Mth pixel to a last pixel in 
each of horizontal pixel lines of the image, a second temporary storage (reading out) 
section adapted to readably store the pixels ranging from a head pixel to a (M-l)th 

2 5 pixel in each of the horizontal pixel lines of the image, and a third temporary storage 
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section adapted to delay outputs of the pixels stored in the first temporary storage 
section, to receive the pixels from the second temporary storage section, and to 
simultaneously output the adjacent pixels consisting of the M pixels adjoining in the 
horizontal direction and the N pixels adjoining in the vertical direction to the 
5 arithmetic section. 

[0009] The image includes both one frame of image and an image block obtained by 
dividing one frame of image. 

[0010] The image processing apparatus further compriooQ includes a control section. 
Befege -Prior to t he pixels for generating a last output pixel of one of the horizontal 

1 0 pixel lines €H?e-being_stored in the third temporary storage (reading out) section, the 
control section reads the pixels ranging from the head pixel to the (M-l)th pixel of the 
next horizontal pixel line fi'om the image storage section to write them to the second 
temporary storage section. After the pixels for generating the last output pixel of the 
one of the horizontal pixel lines io outputto d are output from the third temporary 

15 storage section to the arithmetic section, the control section reads the pixels ranging 
firom the head pixel to the (M-l)th pixel of the next horizontal pixel line fi-om the 
second temporary storage section to write them to the third temporary storage section. 
[0011] At line updatingj ii.e., when, after the generation of the last output pixel in one 
horizontal pixel Une, the first output pixel in the next horizontal pixel line is generated 

20 by the arithmetic section^, it is necessary that the pixels ranging from the head pixel to 
the Mth pixel in the next horizontal pixel line hae -have alreadv been stored in the third 
temporary storage section. In the first embodiment of the present invention, the pixels 
ranging firom the head pixel to the (M"l)th pixel in the next horizontal pixel line ie-are 
stored in the second temporary storage section before line updating and transferred 

2 5 from the second temporary storage section to the third temporary storage section. 



5 

Therefore, even at the hne updating, the third temporary storage section can output a 
set of effective pixels required for the-generating the output pixel to be received by the 
arithmetic section. In other words, processing loss at the line updating can be 
eliminated. 

5 [0012] Further, it is preferable that the N is an integer of two or more, and that with 
respect to the pixels belonging t ofr om the first to the Nth horizontal pixel lines of the 
image, the control section repeats an operation of sequentially reading the N pixels 
arranged in the vertical direction from the image storage section and writing the pixels 
in the first temporary storage section or the second temporary storage section with 

1 0 shifting a reading position in the horizontal direction. 

[0013] At image updatingr (i.e., when, after the generation of the last output pixel in 
the last horizontal pixel hne of one image, the output pixel in the first horizontal pixel 
Une of the next image is generated), it is necessary that the pixels boloncdng t of rom the 
fiirst to the Nth horizontal pixel hnes of the next image has -have b een stored in the 

15 third temporary storage section. By reading the pixels ranging from the first to the 
Nth horizontsd pixel lines of the image and writing the pixels in the second and third 
temporary storage sections as described above, the third temporary storage section can 
output a set of effective pixels required for generating the output pixel to the 
arithmetic section even at the image updating,. In other words, processing loss at the 

2 0 image updating can be ehminated. 

[0014] Furthermore, it is preferable that the arithmetic section generates the output 
pixel in every unit time , that the first temporary storage section has N RAMs, and that 
the control section reads one pixel from the image storage section and writes the pixel 
to the RAM or the second temporary storage section within the unit time when a 

2 5 previous writing address of the RAM under writin g being written to is immediately 
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before a previous reading address . It is also preferable that the a rithmeti c section^ ^anA 
reads plural pixels from the image storage section and writes the pixels to the RAM 
and/or the second temporary storage section within the unit when the previous writing 
address of the RAM under writing is an address two or more immediately before the 
5 previous reading address. 

[0015] Such controlling of the writing address and the reading address of the RAMs 
achieves effective usage of the storage capacities of the RAMs without cavising 
problems in image processing. 

[0016] A second aspect of the present invention provides an image processing method 

10 of generating output pixels by arithmetically processing adjacent pixels consisting of M 
pixels adjoining in a horizontal direction and N pixels adjoining in a vertical direction 
in an image stored in an image storage section, M being an integer of two or more, and 
N being an integer of one or more . The method includin g , comprioinp reading the 
pixels from the image storage section with e^tingshifting a reading position in the 

1 5 horizontal direction and writing the pixels in a first temporary storage section, reading 
the pixel firom the first temporary storage section and writing the pixel in a third 
temporary storage section, delaying output of the pixels by the third temporary storage 
section so as to output the pixels consisting of the M adjacent pixels in the horizontal 
direction and the N adjacent pixels in the vertical direction to an arithmetic section, 

2 0 generating the output pixel from the M X N pixels by the arithmetic sectioni before the 
pixels for generating a last output pixel in one horizontal pixel line is -are stored in the 
third temporary storage section, reading pixels ranging from a head pixel to a (M-l)th 
pixel in the next horizontal pixel Une from the image storage section and writing the 
pixels in a second temporary storage section, and^ after the pixels for generating the 

2 5 last output pixel in the one horizontal pixel line are outputted from the third 
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temporary storage section to the arithmetic section, reading the pixels ranging from 
the head pixel to the (M-l)th pixel in the next line from the second temporary storage 
section and writing the pixels in the third temporary storage section. 
[0017] According to the present invention, the pixels ranging from the head pixel to 
5 the (M-l)th pixel in each horizontal pixel line are previously stored in the second 
temporary storage section and then read and stored in the third temporary storage 
section before the Une updating, whereby processing loss at line updating can be 
eliminated, and processing speed can be improved and power consumption can be 
reduced. Further, the plural pixels belonging to from the first to the Nth horizontal 

1 0 pixel Unes of the image are read in the vertical direction with shifting the reading 

position in the horizontal direction. Therefore, processing loss at the image updating 
can be ehminated, processing speed can be improved, and power consumption can be 
reduced. Furthermore, by changing the speed of writing pixels in the RAMs serving as 
the first storage section depending on the distance between the writing address and the 

1 5 reading address, the storage capacities of the RAMs can be used effectively without 
causing problems in image processing. 

Brief Description of Drawings 

[0018] Other objects and characteristics of the present invention will be clarified 
2 0 according to the following descriptions regarding preferred embodiments referring to 
the accompanying drawings. 

[0019] Fig. 1 is a block diagram showing a digital camera image processing system 
having a YC processing circuit according to an embodiment of the present invention; 
Fig. 2 is a flowchart illustrating the operation of the image processing system shown 
25 in Fig. l; 
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Fig. 3 is a circuit diagram showing the YC processing circuit; 
Fig. 4 is a view showing one frame of image; 
Fig. 5 is a view showing the assignment of coordinates to pixels; 
Fig. 6 is a schematic view showing the structiire of pixel data; 
Fig. 7 is a schematic view showing a RAM; 

Figs. 8A to 8C are views for illustrating a method of generating the output pixels; 
Fig. 9 is a view illustrating the writing destination of the pixel data of each pixel in 
an image block; 

Fig. 10 is a view showing the sequence of writing the pixel data of pixels in an image 
block; 

Fig. 11 is a view showing the sequence of outputting the pixel data of output pixels; 

Fig. 12 is a flowchart for illustrating the operation of the YC processing circuit; 

Fig. 13 is a flowchart of a subroutine at step S12-1 inrof Fig. 12; 

Fig. 14 is a table showing conditions for determining a loading method for first and 
second temporary storage sections; 

Figs. 15 and 16 are tables for illustrating the operation of the YC processing circuit 
from time "t" to time t+143; 

Fig. 17 is a view showing times when the pixel data of respective pixels is 
transferred from a memory to the RAMs; 

Fig. 18 is a view showing times when the pixel data stored in the RAMs is 
transferred to a third storage section; 

Fig. 19 is a view showing times when the pixel data of respective pixels is generated; 

Fig. 20 is a schematic view showing pixel data stored in the first and second 
temporary storage sections immediately after time "t"; 

Fig. 21 is a schematic view showing pixel data stored in the first and second 



temporary storage sections after pixel data is read firom the memory to the RAMs at 
time t+l>* 

Fig. 22 is a schematic view showing pixel data stored in the third temporary storage 
section before the shifting of the registers of the third temporary storage section at 
5 time t+li 

Fig. 23 is a schematic view showing pixel data stored in the third temporary storage 
section after the shifting of the registers of the third temporary storage section at time 
t+i; 

Fig. 24 is a schematic view showing pixel data stored in the third temporary storage 
1 0 section after the pixel data of pixels is transferred pixel by pixel, from the RAMs to the 
third temporary storage section at time t+ 1; 

Fig. 25 is a schematic view showing pixel data stored in the third temporary storage 
section when the pixel data of an output pixel is generated at time t+i; 

Fig. 26 is a schematic view showing pixel data stored in the first and second 
1 5 temporary storage sections immediately after time t+2; 

Fig. 27 is a schematic view showing pixel data stored in the first and second 
temporary storage sections after pixel data is read from the memory to the RAMs at 
time t+3; 

Fig. 28 is a schematic view showing pixel data stored in the third temporary storage 
2 0 section after pixel data is transferred from the second temporary storage section to the 
third temporary storage section; 

Fig, 29 is a schematic view showing pixel data stored in the third temporary storage 
section after the pixel data of pixels is transferred pixel by pixel, from the RAMs to the 
third temporary storage section at time t+3»' 
2 5 Fig. 30 is a schematic view showing pixel data stored in the first and second 
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temporary storage sections immediately after time t+3; 

Fig. 31 is a schematic view showing pixel data stored in the first and second 
temporary storage sections after pixel data is read fi'om the memory to the RAMs at 
time t+4; 

5 Fig. 32 is a schematic view showing pixel data stored in the third temporary storage 

section after an output pixel is generated at time t+4; 

Fig. 33 is a schematic view showing pixel data stored in the first and second 
temporary storage sections immediately after time t+4; 
Fig. 34 is a schematic view showing pixel data stored in the first and second 
1 0 temporary storage sections after pixel data is read firom the memory to the RAMs and 
registers at time t+5; 

Fig. 35 is a schematic view showing pixel data stored in the third temporary storage 
section after the output pixel is generated at time t+5; 

Fig. 36 is a schematic view showing pixel data stored in the first and second 
1 5 temporary storage sections immediately after time t+9; 

Fig. 37 is a schematic view showing pixel data stored in the first and second 
temporary storage sections after pixel data is read from the memory to the RAMs and 
registers at time t+lO; 

Fig. 38 is a schematic view showing the sequence of writing the pixel data of pixels 
20 in one frame; 

Fig. 39 is a schematic view showing the sequence of writing the pixel data of each 
image block; 

Fig. 40 is a schematic view illustrating loss during Une processing; and 
Fig. 41 is a schematic view illustrating loss at the image block processing. 
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Boot Mode for Carrying Qu t Petailed Description of the Invention 
[0020] Fig. 1 shows an image processing system 1 for a_digital camera having an YC 
processing circuit 4 according to an embodiment of an image processing apparatus of 
the present invention, and Fig. 3 shows the details of the YC processing circuit 4. This 
5 image processing system 1 comprises an image pickup circuit 2 having a CCD or the 

Uke, a memory 3 formed of a DRAM or the hke, the YC processing circmt 4, an SRAM 5, 
a resolution conversion circuit 6, a compression processing circuit 7 for executing 
compression processing, such as JPEG compression processing, a recording medium 8, 
such as an IC card, and a control circuit 9. 

1 0 [0021] The operation of this image processing system 1 will be described referring to 
Fig. 2. First, at step S2"l, raw data (RGB data) generated by the image pickup circuit 
2 is written to the memory 3. Then, at step S2-2, the YC processing circuit 4 generates 
YC data on the basis of the raw data read from the memory 3, and the generated YC 
data is written to the SRAM 5. When it is judged at step S2-3 that resolution 

1 5 conversion is to be e xecuted, the resolution conversion circuit 6 carries out resolution 
conversion for the YC data at step S2-4. At step S2-5, the YC data subjected to the 
resolution conversion is written to the SRAM 5 and the memory 3. When it is judged 
at step S2-3 that the resolution conversion is not executed, the YC data generated by 
the YC processing circuit 4 is written to the SRAM 5 and the memory 3 at step S2-5. 

20 At step S2-6, the compression processing circuit 7 compresses the YC data stored in the 
SRAM 5. The compressed data generated by the compression processing is written to 
the memory 3. At step S2-7, the compressed data in the memory 3 is written to the 
recording medium 8. 

[0022] In this embodiment, as shown in Fig. 4, with respect to the raw data stored in 
2 5 the memory 3, 1602 pixels in the horizontal direction and 1218 pixels in the vertical 
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direction form one frame of image 11. In the following explanations, the position of 
each pixel 12 in the image 11 is represented by an X coordinate and a Y coordinate as 
shown in Fig. 5. For example, the pixel 12 represented by (67, 2) is a pixel at the 67th 
position from the left end and the second position from the upper end of the image 11. 
Furthermore, in the following explanations, a pixel arrangement or low in the 
horizontal direction is referred to as a Une as necessary. 

[0023] Further, as shown in Figs. 4 and 5, the YC processing circuit 4 divides one 
frame of image 11 into image blocks 13 each formed of 66 pixels adjoining in the 
horizontal direction and 66 pixels adjoining in the vertical direction for processing. As 
described later, an arithmetic circuit 20 generates the YC data of an output pixel from 
the pixel data of nine pixels 12 formed of three pixels 12 adjoining in the horizontal 
direction and three pixels 12 adjoining in the vertical directions. Therefore, the pixel 
data of two pixels 12 in the horizontal direction, positioned at the boundary between 
two image blocks 13 adjoining in the horizontal direction, is used for the processing of 
both the image blocks 13. Similarly, the pixel data of two pixels 12 in the vertical 
direction, positioned at the boundary between two image blocks 13 adjoining in the 
vertical direction, is used for the processing of both the image blocks 13. One frame of 
image 11 includes 25 image blocks 13 in the horizontal direction and 19 image blocks 
13 in the vertical direction, 475 image blocks 13 in total. 

[0024] As shown in Fig. 6, the raw data (hereafter referred to as pixel data) of each 
pixel 12 has an even/odd bit 14b and an effective bit 14c as the accompanying 
information of a pixel data portion 14a. The even/odd bit 14b indicates whether the 
pixel 12 belongs to an even-numbered line or an odd-numbered line. The effective bit 
14c indicates whethe r or not the pixel data portion 14a of the pixel data of the pixel 12 
is effective-er-net. 
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[0025] Referring to Fig. 3, the YC processing drciiit 4 (portions identified bv dotted 
lines) c omprises the arithmetic circuit 20 for arithmetically processing the raw data of 
the pixels 12 stored in the memory 3, and also comprises a first temporary storage 
(reading out) section 21, a second temporary storage (reading out) section 22, a third 
5 temporary storage (reading out) section 23, a first selection section 26 and a second 
selection section 27 provided to output the pixel data of the pixels 12 stored in the 
memory 3 in arithmetically processable combinations. 

[0026] As shown in Figs. 8A to 8C, the arithmetic circuit 20 executes weighing 
addition of the pixel data of nine pixels 12 formed of three pixels 12 adjoining in the 

10 horizontal direction and three pixels 12 adjoining in the vertical direction. For 

example, as shown in Fig. 8A, by weighing addition of the pixel data of nine pixels 12 
from (1, 1) to (3, 3), the pixel data of the output pixel (l, l) is obtained. For executing 
this weighing addition, the arithmetic circuit 20 has multipUers 31a, 31b and 31c, and 
an adder 32 for adding the outputs of these multipUers 31a to 31c. The arithmetic 

1 5 circmt 20 generates the pixel data of one output pixel in every unit time. 

[0027] The first temporary storage section 21 has three RAMs 21a, 21b, and 21c, each 
of which can store pixel data read from the memory 3. As described later in detail, the 
pixel data of the pixels 12 ranging fi'om the third pixel fi'om the head pixel to the last 
pixel (the 66th pixel) in each Une in each image block 13 is stored in these three RAMs 

20 21a, 21b, and 21c. As shown in Fig. 7, each of the RAMs 21a, 21b, and 21c has 

addresses 33, fi-om No. 1 to No. 64. In each address 33, the pixel data of one pixel can 
be readably stored. 

[0028] The second temporary storage section 22 has six registers 22ad, 22ae, 22bd, 
22be, 22cd and 22ce, each of which can store the raw data of one pixel 12. As described 
2 5 later in detail, these six registers 22ad to 22ce are used to store the pixel data of the 
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pixels 12 ranging from the head (first) pixel 12 to the second pixel 12 in each line in 
each image block 13. The registers 22ad and 22ae correspond to the RAM 21a of the 
first temporary storage section 21. Further, the registers 22bd and 22be correspond to 
the RAM 21b. Furthermore, the registers 22cd and 22ce correspond to the RAM 21c. 
5 [0029] The third temporary storage section 23 has nine registers 23aa, 23ab, 23ac, 
23ba, 23bb, 23bc, 23ca, 23cb, and 23cc, each of which can store the pixel data of one 
pixel 12. The registers 23aa to 23ac correspond to the RAM 21a of the first temporary 
storage section 21. Further, the registers 23ba to 23bc correspond to the RAM 21b. 
Furthermore, the registers 23ca to 23cc correspond to the RAM 21c. 

1 0 [0030] The setting of the first selection section 26 can be changed between the upper 
position indicated by soUd lines and the lower position indicated by dotted Unes. When 
the setting of the first selection section 26 is at the upper position, every three of the 
registers 23aa to 23cc are connected in series. Specifically, the registers 23aa to 23ac, 
the registers 23ba to 23bc, and the registers 23ca to 23cc are respectively connected in 

1 5 series. Therefore, when the setting of the first selection section 26 is at the upper 

position, the registers 23aa to 23ac. the registers 23ba to 23bc, and the registers 23ca 
to 23cc of the third temporary storage section 23 respectively delay the output of pixel 
data from the RAMs 21a to 21c, and then output pixel data to the arithmetic circtiit 20 
via the second selection section 27. 

2 0 [0031] When the setting of the first selection section 26 is at the lower position, the 
third temporary storage section 23 is connected to the second temporary storage 
section 22. Specifically, the registers 22ad and 22ae of the second temporary storage 
section 22 are respectively connected to the registers 23ab and 23ac of the third 
temporary storage section 23. Further, the registers 22bd and 22be of the second 

2 5 temporary storage section 22 are respectively connected to the registers 23bb and 23bc 
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of the third temporary storage section 23. Furthermore, the registers 22cd and 22ce of 
the second temporary storage section 22 are respectively connected to the registers 
23cb and 23cc of the third temporary storage section 23. Therefore, when the setting of 
the first selection section 26 is at the lower position, pixel data is respectively output 
5 firom the registers 22ad and 22ae, the registers 22bd and 22be, and the registers 22cd 
and 22ce of the second temporary storage section 22 to the registers 23ab and 23ac, the 
registers 23bb and 23bc, and the registers 23cb and 23cc of the third temporary storage 
section 23. 

[0032] The second selection section 27 has a function of changing the connection 
1 0 between the third temporary storage section 23 and the arithmetic circuit 20 so that 

the pixel data of the latest line is input to the mvdtiplier 31a of the arithmetic circuit 20, 
the pixel data of the latest Une but one is input to the multipUer 31b, and the pixel 
data of the latest line but two is input to the multipher 31c. 

[0033] The control circuit 9 controls the memory 3, the RAMs 21a to 21c, the registers 
15 22ad to 22ce, the registers 23aa and 23cc, and the first selection section 26 and the 
second selection section 27 to transfer pixel data from the memory 3 to the first 
temporary storage section 21 and the second temporary storage section 22, to transfer 
pixel data firom the first temporary storage section 21 and the second temporary 
storage section 22 to the third temporary storage section 23, and to transfer pixel data 
2 0 firom the third temporary storage section 23 to the arithmetic circuit 20. In addition, 
the control circuit 9 controls the arithmetic circuit 20 to execute arithmetic processing 
and to transfer generated pixel data to the SRAM 5. 

[0034] Further, pixel data is input to the clock control section 9a of the control circmt 
9 via the third temporary storage section 23. On the basis of the even/odd bit 14b and 
2 5 the effective bit 14c, the clock control section 9a makes a judgment as to whether or not 
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the arithmetic circuit 20 can generate effective output pixel data from the pixel data 
stored in the third temporary storage section 23 or not . If the clock control section 9a 
judges that the arithmetic circuit 20 can not generate effective output pixel data from 
the pixel data stored in the third temporary storage section 23, the clock control section 
5 9a stops the supply of its clock signal to the arithmetic circuit 20, thereby inhibiting 
the arithmetic processing of the arithmetic circuit 20. This prevents wasteful power 
consumption at-bjLthe arithmetic circuit 20 when ineffective pixel data is input. 
[0035] Then, the YC processing circuit 4 divides one frame of image 11 into 475 image 
blocks 13 in total, each formed of 66 pixels adjoining in the horizontal direction and 66 

1 0 pixels adjoining in the vertical direction, and processes the image blocks 13. The 
sequence of processing the image blocks 13 is indicated by arrow "A" in Fig. 4. 
Specifically, the processing starts from the image block 13 located at the upper left 
corner of the image 11, and the image blocks 13 are processed sequentially, line by Une. 
[0036] The processing executed for each image block 13 by the YC processing drcviit 4 

1 5 can be broadly divided into three stages. At a first stage, the pixel data read from the 
memory 3 is written to the first temporary storage section 21 or the second temporary 
storage section 22. At a second stage, the pixel data read from the first temporary 
storage section 21 or the second temporary storage section 22 is written to the third 
temporary storage section 23. At a third stage, the arithmetic circuit 20 generates 

2 0 output pixel data on the basis of the pixel data read from the third temporary storage 
section 23. As described latter in detail, in accordance with this embodiment, at the 
second stage, time loss during Une updating is eUminated by providing the second 
temporary storage section 22. 

[0037] The writing destinations of the pixel data read from the memory 3 at the first 
25 stage are shown in Fig. 9. Specifically, the pixel data of the first two pixels 12, i.e., the 
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first and second pixels in each line is written to the second temporary storage section 
22. On the other hand, the pixel data of the third to the last (66th) pixels 12 in each 
hne is written to the first temporary storage section 21. 

[0038] The sequence of reading the pixel data of the pixels 12 belonging to each image 
5 block 13 fi"om the memory 3 and writing the pixel data in the first temporary storage 
section 21 or the second temporary storage section 22 at the first stage is shown in Fig. 
10. Specifically, the pixel data of the pixels ranging from the fourth Une to the 66th 
line in the image block 13 is written to the first temporary storage section 21 or the 
second temporary storage section 22 with shifting the reading position, pixel by pixel, 

10 in the horizontal direction in every line as indicated by an arrow "B". On the other 
hand, with respect to the pixel data of the pixels 12 ranging from the first Hne to the 
third line in the image block 13, the operation of sequentially reading the pixel data of 
the three pixels 12 arranged in the vertical direction from the memory 3 and writing 
the pixel data in the first temporary storage section 21 or the second temporary storage 

15 section 22 is repeated with shifting the reading position in horizontal direction as 
indicated by an arrow "C". As described later in detail, in accordance with this 
embodiment, time loss at the image block updating is eliminated by reading the pixel 
data in the vertical direction instead of the horizontal direction at the start of the 
processing of each image block 13. 

2 0 [0039] At the first stage, the sequence of reading the pixel data from the memory 3 
and writing the pixel data in the corresponding RAMs 21a to 21c is shifted along the 
addresses 33 of the RAMs 21a to 21c in the horizontal direction as indicated by an 
arrow "D" in Fig. 7. 

[0040] At the second stage, the sequence of reading the pixel data from the RAMs 21a, 
25 21b, and 21c of the first temporary storage section 21 and writing the pixel data in the 
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corresponding registers 23aa, 23ba. and 23ca of the third temporary storage section 23 
is shifted along the addresses 33 of the RAMs 21a to 21c in the horizontal direction as 
indicated by the arrow "D" in Fig. 7. 

[0041] Referring to Fig. 7, the relationship between the reading point and the writing 
point in each of the RAMs 21a, 21b, and 21c of the first temporary storage section 21 
will be described. As described above, each of the RAMs 21a, 21b, and 21c has 
addresses No. 1 to No. 64. The reading point is an address of each of the RAMs 21a to 
21c from which pixel data is read. Further, the writing point is an address of each of 
the RAMs in which pixel data newly read from the memory 3 is written. 
[0042] In each of the RAMs 21a to 21c, the present writing point cannot be beyond 
the previous reading point. For example, when the previous reading point is address 
No. 61 as shown in Fig. 7, the writing point is required to be address No. 60 or before 
the address No. 60. This is because that the writing point overtaking or being beyond 
the previous reading point means that new pixel data read from the memory is 
overwritten to addresses 22 of RAMs 21a to 21c in which pixel data that has not yet 
used for generating output pixel data is stored. 

[0043] The writing point may be delayed from the reading point as long as that the 
writing point is not beyond the reading point. For example, when the reading point is 
address No. 61 as shown in Fig. 7, the writing point shoTold be address No. 60 or before 
the address No. 60. However, it is preferable that the writing point is close to the 
reading point. Ideally, it is preferable that the writing point is immediately before the 
reading point. For example, when the reading point is the address No. 61 as shown in 
Fig. 7, it is ideal that the writing point is the address No. 60. The writing point 
immediately before the reading point means that, when pixel data is output from an 
address 33 of any one of the RAMs 21a to 21c to the third temporary storage section 23, 
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overwriting with new pixel data read from the memory 3 is immediately executed at 
the address 33. In this status, a margin in which the writing point can be delayed from 
the reading point becomes maximum, whereby the storage capacities of the RAMs 21a 
to 21c are used most effectively. 
5 [0044] The speed of reading pixel data from the memory 3 and writing the pixel data 
in the RAMs 21a to 21c of the first temporary storage section 21 and the registers 22ad 
to 22ce of the second temporary storage section 22 is determined on the basis of 
conditions regarding the writing point and the reading point described above. 
Specifically, when the previous writing point in one of the RAMs in which data is being 

1 0 written is immediately before the previous reading point, the pixel data of one pixel 12 
is written to one of the RAMs 21a to 21c and the registers 22ad to 22ce in the unit time. 
On the other hand, when the previous writing point in one of the RAMs in which data 
is being written is not immediately before the reading point, the pixel data read from 
the memory 3 is written in any two of the RAMs 21a to 21c or both of any one of the 

15 RAMs 21a to 21c and any one of the registers 22ad to 22ce in the unit time. 

[0045] The above-mentioned address control in the RAMs 21a to 21c and the above- 
mentioned adjustment of the writing speed, i.e., number of pixels per unit time written 
to the RAMs 21a to 21c and the registers 22ad to 22ce, are executed by the control 
circuit 9. 

2 0 [0046] At the third stage, the sequence of generating output pixel data and 

outputting the pixel data to the SRAM 5 by the arithmetic circuit 20 is shown in Fig. 11. 
Specifically, as indicated by an arrow "E", the generated output pixel data is moved, 
pixel by pixel, in the horizontal direction in every image block 13. Since the arithmetic 
circuit 20 generates output pixel data from the pixel data of three pixels 12 adjoining in 

2 5 the horizontal direction and three pixels 12 adjoining in the vertical direction, the 
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number of pixels included in an outputted image block 13* is smaller than that of pixels 
included in the image block 13 (see Fig. 4) of the image 11 stored in the memory 3. 
Specifically, 64 output pixels in the horizontal direction and 64 output pixels in the 
vertical direction, i.e., 4096 output pixels, are included in the image block 13*. 
5 [0047] Then, the operation of the YC processing circvdt 4 will be described. Fig. 12 is 
a flowchart showing the operation of the YC processing circvdt 4 carried out in every 
unit time. This operation is executed by the control circuit 9, Referring to Fig. 12, step 
S12-1 corresponds to the above-mentioned first stage, steps S12-2 to 812*5, and steps 
S12-7 and S12-8 correspond to the above-mentioned second stage, and step S12-6 
10 corresponds to the above-mentioned third stage. The arithmetic circuit 20 generates 
one output pixel in every unit time as described above. 

[0048] At step S12-1 in Fig. 12, a method of reading pixel data firom the memory 3 is 
determined, and pixel data reading is executed on the basis of the determined reading 
method. Specifically, the reading speed of pixel data, the writing destinations of pixel 

15 data having been read, and the reading direction of pixel data are determined. 

[0049] Referring to Fig. 13. first, at step S13-1, a judgment is made as to whether or 
not the previous writing point in one of the RAMs 21a to 21c under writing is 
immediately before the previous reading point »-net-(condition l). If the previous 
writing point is immediately before the previous reading point, the number of pixels 

2 0 read out in a unit time is determined to be one at step S13-2; if not immediately before 
the previous reading point, the number of pixels read out in the unit time is 
determined to be two at step S13-3. In this way, the reading speed is determined on 
the basis of the distance between the writing point and the reading point. 
[0050] Then, at step S13-4, a judgment is made as to whether or not pixel data 

2 5 required for generating the first output pixel in the next line has been stored in the 
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registers 22ad to 22ce of the second temporary storage section 22 (condition 2). If the 
pixel data has been stored, the pixel data read from the memory 3 is written to the 
RAMs 21a to 21c of the first temporary storage section 21 at step S13-5. On the other 
hand, if the pixel data has not been stored, the pixel data read from the memory 3 is 
5 written to the registers 22ad to 22ce of the second temporary storage section 22, or 
written to the registers 22ad to 22ce and the RAMs 21a to 21c of the first temporary 
storage section 21 at step S13-6. In this way, the writing destinations of pixel data are 
determined on the basis of pixel data stored in the second temporary storage section 22. 
[0051] Then, at step S13-7, a judgment is made as to whether or not the pixel data 

1 0 read firom the memory 3 and written in the RAMs 21a to 21c of the first temporary 
storage section 21 and/or the registers 22ad to 22ce of the second temporary storage 
section 22 corresponds to the pixel belonging to the first three lines of the image block 
13 (condition 3). If the pixel data does not correspond to the pixel belonging to the first 
three Unes, the reading direction is determined to be horizontal at step S13-8. On the 

1 5 other hand, if the pixel data corresponds to the pixel belonging to the first three Hnes, 
the reading direction is determined to be vertical at step S13-9. In this way, the 
reading direction is determined on the basis of whether or not the pixel 12 ivom which 
pixel data is being read is a pixel in the first lines of the image block. 
[0052] Fig. 14 shows the relationship among conditions 1, 2, and 3 and reading 

2 0 methods, i.e., loading methods. Since there are two kinds for each of the reading speed, 
writing destination, and reading direction (loading direction), eight kinds of loading 
methods A to H exist in total. For example, in case that the previous writing point in 
one of the RAMs 21a to 21c under writing is immediately before the previous reading 
point (condition l), that pixels required for generating the first output pixel in the next 

2 5 Une has been stored in the second temporary storage section 22 (condition 2), and that 
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the pixel 12 previously read from the memory 3 and written to the RAMs 21a to 21c or 
the Hke is not the pixel 12 belonging to the first three Unes of the image block 13 
(condition 3), the loading method A is adopted so that one pixel 12 is loaded in the 
horizontal direction from the memory 3 to the RAMs 21a to 21c. 
5 [0053] After reading from the memory 3 is executed at step S12-1, a judgment is 
made at step S12-2 as to whether or not the previously generated output pixel is the 
last output pixel in a Une in the image block 13\ that is, a judgment is made as to 
whether or not there comes a_time when the line updating is executed. If the output 
pixel is not the last pixel in the Une. that is, if it is not tiHafflg-atime.for line updating, 
1 0 then the processing advances to step S12-7. On the other hand, if it is timing atime 
for Une updating, the processing advances to step S12-3. 

[0054d When Une updating is not executed, steps S12-7 and S12-8 are carried out. 
First, at step S12-7, the pixel data of the third temporary storage section 23 is shifted 
from one register to another. For example, in the registers 23aa to 23cc, the pixel data 

15 in the register 23aa is shifted to the register 23ab, and the pixel data in the register 
23ab is shifted to the register 23ac. Then, at step S12-8, the pixel data stored in the 
first temporary storage section 21 is read and stored in the third temporary storage 
section 23. SpedficaUy, pixel data is read from the RAMs 21a to 21c and written to the 
registers 23aa, 23ba, and 23ca. 

2 0 [0055] Next, at step S12-6, the pixel data stored in the third temporary storage 

section 23 is outputted to the arithmetic circuit 20 via the second selection section 27. 
More specifically, the pixel data is outputted from the nine registers 23aa to 23cc of the 
third temporary storage section 23 to the multipUers 31a to 31c of the arithmetic 
circuit 20. The arithmetic circuit 20 generates an output pixel from the inputted pixel 

2 5 data. 
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[0056] At the line updating, steps S12-3 to S12-5 are executed. First, at step S12-3, 
the setting of the first selection section 26 is changed to the lower position, whereby the 
pixel data stored in the second temporary storage section 22 is read and stored in the 
third temporary storage section 23. Specifically, the pixel data is respectively 
5 outputted from the registers 22ad and 22ae to the registers 23ab and 23ac, from the 
registers 22bd and 22be to the registers 23bb and 23bc, or from the registers 22cd and 
22ce to the registers 23cb and 23cc. Next, at step S12-4, the pixel data stored in the 
first temporary storage section 21 is read and stored in the third temporary storage 
section 23. Specifically, the pixel data is read from the RAMs 21a to 21c and written to 

10 the registers 23aa, 23ba, and 23ca. Next, at step S12-5, the setting of the second 

selection section 27 is changed. Specifically, the setting of the second selection section 
27 is changed so that the pixel data of the latest line is inputted to the multiplier 31a of 
the arithmetic circuit 20, so that the pixel data of the latest line but one is input to the 
multiplier 31b, and so that the pixel data of the latest line but two is input to the 

15 mxdtiplier 31c. Then, at step S12-6, the pixel data stored in the third temporary 

storage section 23 is output to the arithmetic circuit 20 via the second selection section 
27, and the arithmetic circuit 20 generates an output pixel from the inputted pixel data. 
[0057] Figs. 15 to 19 show examples obtained when the YC processing circuit 4 
executes YC processing for the pixel data stored in the memory 3 by repeating the 

2 0 processing shown in the flowchart of Fig. 12 at the unit time intervals. 

[0058] First, Figs. 15 and 16 show the operation statuses of the YC processing circuit 
4 at respective times from time t, at which the output pixel (62, 63) in one frame of 
image is generated to time t+143. In Figs. 15 and 16, an item 1 represents a method of 
loading pixel data from the memory 3 to the RAMs 21a to 21c and the registers 22ad to 

2 5 22cd. An item 2 represents the writing destination of pixel data read from the memory 
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3. An item 3 represents a pixel written from the memory 3 to the RAMs 21a to 21c and 
the like. Items 4 and 5 respectively represent the writing destination of pixel data and 
a pixel whose pixel data is read when the two pixels 12 ie -are r ead in the unit time. 
Items 6 and 7 respectively represent the writing point and reading point of the RAMs 
5 21a to 21c at each time (see Fig. 7). An item 8 represents the selection of setting status 
of the first selection section 26. An item 9 represents an output pixel. 
[0059] Fig. 17 shows times at which pixel data is read from the memory 3 and written 
to the RAMs 21a to 21c of the first temporary storage section 21 or the registers 22ad 
to 22ce of the second temporary storage section 22 by the processing at step S12-1 in 

1 0 Fig. 12. Further, Fig. 18 shows times at which pixel data is read from the RAMs 21a to 
21c and written to the registers 23aa, 23ba, and 23ca of the third temporary storage 
section 23 by the processing at steps S12-4 and S12-8 in Fig. 12. Furthermore, Fig. 19 
shows times at which output pixel data is generated by the processing at step S12-6 in 
Fig. 12. In these Figs. 17 to 19, the numerals in the blocks indicating pixels 12 

15 represent times. For example, the numeral "4" shown in the pixel 12 (65, l) in Fig. 17 
indicates that the pixel data of this pixel is read from the memory 3 at time 4. Further, 
Figs. 17 to 19 shows time "1" at which the pixel data of the output pixel (63, 63) 
belonging to the first image blocks 13a and 13a* (see Figs. 4 and 11) is generated to 
time tl44 at which the pixel data of the output pixel (78, 2) belonging to the second 

2 0 image blocks 13b and 13b' is generated. In Figs. 17 to 19, the numerals in the blocks 
represent times. The times in the respective pixels in Figs. 17 to 19 correspond to the 
cases wherein "t" is 0 in Figs. 15 and 16. 

[0060] As shown in Fig. 18, pixel data writing from each of the RAMs 21a to 21c of 
the first temporary storage section 21 to the third temporary storage section 23 merely 
2 5 advances in the horizontal direction pixel by pixel. As shown in Fig. 19, the generation 
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of output pixel by the arithmetic circuit 20 also advances in the horizontal direction 
pixel by pixel. On the other hand, pixel data writing from the memory 3 to each of the 
RAMs 21a to 21c of the first temporary storage section 21 or the registers 22ad to 22ce 
of the second temporary storage section 22 (at step S12-1 in Fig. 12) is executing 
5 according to Figs. 13 and 14, whereby the reading direction is changed as shown in Fig. 
17, and the reading speed and reading destination are also changed. 
[0061] According to steps S13-4 to S13-6 in Fig. 13 and the condition 2 in Fig. 14, 
when pixel data required for generating the pixel data of the first output pixel in the 
next Une has not been stored in the second temporary storage section 22, the pixel data 

10 of head and second pixels 12 in the next Une is stored in the registers 22ad to 22ce of 

the second temporary storage section 22. For example, in Figs. 15 to 17, at times 4 to 9 
and times 68, 69, 132, and 133, the pixel data of the first or second pixel 12 in a line 
read from the memory 3 is written to the registers 22ad to 22ce instead of the RAMs 
21a to 21c corresponding to the line. In the case of line updating at step S12-2 in Fig. 

15 12, the pixel data stored in the registers 22ad to 22ce is written to the third temporary 
storage section 23 at step S12-3. The pixel data written from the second temporary 
storage section 22 to the third temporary storage section 23 is output to the arithmetic 
circuit 20 at step S12-6, together with the pixel data of the third pixel in the next line, 
which is written from the first temporary storage section 21 to the third temporary 

2 0 storage section 23 at step S12-4. whereby all the pixel data reqxiired for the generation 
of output pixel data is supplied to the arithmetic circuit 20. As described above, time 
loss at the Hne updating is ehminated by providing the second temporary storage 
section 22 in addition to the RAMs 21a to 21c of the first temporary storage section 21 
and by storing the pixel data of the first and second pixels in the next line in the second 

2 5 temporary storage section 22 in advance before the line updating. 
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[0062] On the basis of step S13-7 in Fig. 13 and the condition 3 in Fig. 14. the 
direction of reading pixel data from the memory 3 to the first temporary storage section 
21 and the second temporary storage section 22 is changed. At step S13-7, when the 
pixel data previously written from the memory 3 to the first temporary storage section 
5 21 or the second temporary storage section 22 does not correspond to the pixel 12 

within the first three hnes, the pixel data of the pixels in the two lines among the pixel 
data required for generating the pixel data in the next Une has already been stored in 
the RAMs 21a to 21c. Therefore, maintaining reading direction in horizontal direction 
in this case does not cause loss at the image block updating. However, at step Si 3-7, 

1 0 when the pixel data previously written from the memory 3 to the first temporary 

storage section 21 or the second temporary storage section 22 corresponds to the pixel 
12 within the first three lines, the generation of the pixel data of the pixels in the next 
line corresponds to the generation of the pixel data of the pixels in a Une of the next 
image block 13, and the pixel data required for generating the pixel data in the next 

1 5 fine has not been yet stored in the RAMs 21a to 21c at all. Therefore, if the reading 

direction remains horizontal, loss occurs at the image block updating. In this case, the 
reading direction is changed to vertical. By setting the reading direction to vertical in 
the first three or less Unes of the image block 13 as described above, after the last 
output pixel of the previous image block 13 is generated, the first output pixel of the 

2 0 next image block 13 can be generated immediately, whereby the time loss at the image 
block updating can be reduced. For example, as shown in Fig. 19, the last output pixel 
(64, 64) of the first image block 13a' (see Fig. 11) is generated at time 66, and the first 
output pixel (65, l) of the second image block 13' is generated at time 67, immediately 
after time 66. This is achieved by reading the pixel data of the pixels 12 belonging to 

2 5 the first three Unes of the second image block 13' at time 4 and thereafter as shown in 
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Fig. 17. 

[0063] The speed of reading pixel data from the memory 3 and writing the pixel data 
to the RAMs 21a to 21c or the registers 22ad to 22ce is changed to one pixel per unit 
time or two pixels per unit time according to steps S13-1 to S13-3 in Fig. 13 and the 
5 condition 1 in Fig. 14 depending on the distance between the writing point and the 
reading point in the RAMs 21a to 21c. When the writing point is immediately before 
the reading point at step S13-1 in Fig. 13, it is an ideal status wherein the storage 
capacities of the RAMs 21a to 21c are used most effectively as described above. 
Therefore, pixel data is read from the memory 3 to the RAMs 21a to 21c at a rate of one 

1 0 pixel per unit time (at the same speed as the output pixel data generation speed at the 
arithmetic circuit 20) so that the writing point does not become closer to the reading 
point. On the other hand, when the writing point is not immediately before the 
reading point at step S13-1 in Fig. 13, the writing point is delayed from the reading 
point. Therefore, pixel data is read from the memory 3 to the RAMs 21a to 21c at a 

1 5 rate of two pixels per unit time (at the speed of %wiee -two times ae-that of the output 
pixel data generation at the arithmetic circuit 20) so that the writing point catches up 
to the reading point. Although the image block 13 has 66 pixels in the horizontal 
direction, each of the RAMs 21a to 21c has 64 addresses 33 fewer than 66. However, by 
controlling the speed of writing to the RAMs 21a to 21c so that the writing point does 

2 0 not become beyond the reading point and so that the writing point becomes close to the 
reading point, the storage capacities of the RAMs 21a to 21c are used effectively, 
whereby such that p rocessing does not cauoing cause time loss at the above-mentioned 
line updating and image block updating can be attained. 

[0064] Next, referring to Fig. 12 and Figs. 20 to 37, the operation of the YC processing 
2 5 circuit 4 from time t+1 to time t+10 in Fig. 15 (from time 1 to time 10 in Figs. 17 to 19) 
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will be described in detail. 

[0065] Fig. 20 shows the pixel data stored in the RAMs 21a to 21c of the first 
temporary storage section 21 and the registers 22ad to 22ce of the second temporary 
storage section 22 immediately before step S12-1 in Fig. 12. The pixel data of the 
pixels (3, 64) to (66, 64) is stored in the RAM 21a. Further, the pixel data of the pixels 
12 (3. 65) to (66, 65) is stored in the RAM 21b. Furthermore, the pixel data of the 
pixels (64, 63) to (66, 63) and the pixels (3, 66) to (63, 66) is stored in the RAM 21c. The 
pixel data of the six pixels (l, 64) to (2, 66) is stored in the registers 22ad to 22ce. 
[0066] At step S12-1 in Fig. 12, overwriting with the pixel data of the pixel (64, 66) 
read from the memory 3 is carried out for the address 33 of the RAM 21a in which the 
pixel data of the pixel (64, 63) is stored. Fig. 21 shows the pixel data stored in the 
RAMs 21a to 21c and the registers 22ad to 22ce after this overwriting. 
[0067] This overwriting is determined according to Fig. 14. First, with respect to the 
condition 1, the previous writing point of the RAM 21c under writing is the address in 
which the pixel data of the pixel (63, 66) is stored, and the previous reading point is the 
address in which the pixel data of the pixel (64, 63) is stored, whereby the writing point 
is immediately before the reading point. With respect to the condition 2, the pixel data 
of the pixels (l, 64) to (2, 66) required for generating the pixel data of the pixels in the 
next hne (the 64th Une) has already been stored in the second temporary storage 
section 22. With respect to the condition 3, the pixel (64, 66) to be read from the 
memory 3 is not a pixel in the first three fines of the image block 13. Therefore, the 
loading method A is selected, and the pixel data of the pixel (64, 66) is read from the 
memory 3 and written to the RAM 21c. By overwriting with the pixel data of the pixel 
(64, 66), the pixel data of the pixel (64, 63) is erased from the RAM 21c. Since the pixel 
data of this pixel (64, 63) is not used again for the generation of output pixel data, no 
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problem occurs. 

[0068] Fig. 22 shows the status of the pixel data stored in the third temporary storage 
section 23 immediately after the pixel data of the pixel (64, 66) is written from the 
memory 3 to the RAM 21c at step S12-1 of time t+1, that is, immediately before 
5 shifting is carried out at the third temporary storage section 23. The pixel data of the 
pixels (62, 64) to (64, 64) is stored in the registers 23aa to 23ac, the pixel data of the 
pixels (62, 65) to (64, 65) is stored in the registers 23ba to 23bc, and the pixel data of 
the pixels (62, 63) to (64, 63) is stored in the registers 23ca to 23cc. At time "t", the 
arithmetic circuit 20 has already generated new pixel data of the pixel (62, 63) from the 

1 0 pixel data of these nine pixels and has outputted the data. 

[0069] Fig. 23 shows the status of the pixel data stored in the third temporary storage 
section 23 immediately after shifting is carried out at the third temporary storage 
section 23 at step S12-7 of time t+1. The pixel (62, 63) generated previously (at time 
"t") does not correspond to the last pixel (64, 63) in the 63th line of the image block 13 

15 (at step S12-2). Therefore, data in the third temporary storage section 23 is shifted at 
step S12-8. Specifically, the pixel data in the register 23ab is shifted to the register 
23ac and rewritten, and the pixel data in the register 23aa is shifted to the register 
23ab and rewritten. Similarly, pixel data in the other registers 23bb, 23bc, 23cd, and 
23cc is also rewritten. 

2 0 [0070] Fig. 24 shows the status of the pixel data stored in the third temporary storage 
section 23 immediately after step S12-8 of time t+1. At step S12-8, the pixel data of the 
pixel (65, 63) is read from the RAM 21c and written to the register 23ca, the pixel data 
of the pixel (65, 64) is read from the RAM 21a and written to the register 23aa, and the 
pixel data of the pixel (65, 65) is read from the RAM 21b and written to the register 

2 5 23ba. Since the pixel data of the six pixels (63, 63) to (64, 65) has already been stored 



30 

in the third temporary storage section 23 at step S12-7, the pixel data of the nine pixels 
(63, 63) to (65, 65) is thus stored in the third temporary storage section 23. Since this 
is the last time when the pixel data of the pixel (65, 63) is read from the RAM 21c, it is 
not necessary to store the pixel data of the pixel (65, 63) hereafter. In other words, 
5 even if the pixel data of the pixel (65, 63) is erased by overwriting, subsequent image 
processing can be carried out without problems. 

[0071] Fig. 25 shows the output pixel generated at time t+1 and the status of the 
pixel data stored in the third temporary storage section 23, in other words, the status 
immediately after step S12-6. At step S12-6, the pixel data of the nine pixels (63, 63) to 

1 0 (65, 65) stored in the third temporary storage section 23 is input to the arithmetic 

circuit 20 via the second selection section 27, and the arithmetic circuit 20 generates 
the pixel data of the output pixel (63, 63). The pixel data of the pixels in the 64th Une 
is stored in the registers 23aa to 23ac, the pixel data of the pixels in the 65th line is 
stored in the registers 23ba to 23bc, and the pixel data of the pixels in the 63th Une is 

1 5 stored in the registers 23ca to 23cc. Therefore, by the second selection section 27, the 
registers 23aa to 23ac are connected to the multipUer 31b, the registers 23ba to 23bc 
are connected to the multipUer 31c, and the registers 23ca to 23cc are connected to the 
multipUer 31a. 

[0072] The operation at time t+2 is similar to the operation at time t+1. Fig. 26 
2 0 shows the status of the pixel data stored immediately after the pixel data of the pixel 
(64, 63) is generated at time t+2, in other words, the status immediately before step 
S12-1 of time t3. The pixel data of the pixels (3, 64) to (66, 64) is stored in the RAM 
21a, the pixel data of the pixels (3, 65) to (66, 65) is stored in the RAM 21b, and the 
pixel data of the pixel (66, 63) and the pixels (3, 66) to (65, 66) is stored in the RAM 21c. 
25 Further, the pixel data of the pixels (l, 64) to (2, 66) is stored in the second temporary 
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storage section 22. 

[0073] Fig. 27 shows the status of the pixel data stored in the first temporary storage 
section 21 and the second temporary storage section 22 immediately after step S12-1 at 
time t+3. At step S12-1, on the basis of the judgment results of the conditions 1 to 3, 
5 overwriting with the pixel data of the pixel (66, 66) read from the memory 3 is carried 
out for the storage area of the RAM 21c in which the pixel data of the pixel (66, 63) is 
stored. 

[0074] Fig. 28 shows the status of the pixel data stored in the third temporary storage 
section 23 immediately after the pixel data of the six pixels (1, 64) to (2, 66) stored in 

1 0 the second temporary storage section 22 is read and written to the third temporary 

storage section 23 at step S12-3 of time t+3. The pixel (64, 63) generated previously (at 
time t+2) corresponds to the last pixel (64, 63) in the 63th line of the image block 13. 
Therefore, according to step S12-2, pixel data is loaded fi-om the second temporary 
storage section 22 to the third temporary storage section 23 at step S12-3. Specifically, 

15 the setting of the first selection section 26 is changed to the lower position, whereby the 
pixel data in the register 22ad is written to the register 23ab, the pixel data in the 
register 22ae is written to the register 23ac, the pixel data in the register 22bd is 
written to the register 23bb, the pixel data in the register 22be te-is written to the 
register 23bc, the pixel data in the register 22cd is written to the register 23cb, and the 

2 0 pixel data in the register 22ce is written to the register 23cc. 

[0075] Fig. 29 shows the status of the pixel data stored in the third temporary storage 
section 23 immediately after the pixel data of the three pixels (3, 64) to (3, 66) is read 
firom the RAMs 21a to 21c of the first temporary storage section 21 and written to the 
third temporary storage section 23 at step S12-4 of time t+3. After the pixel data in the 

2 5 third temporary storage section 23 has become the status shown in Fig. 27, the setting 
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of the second selection section 27 is changed at step S12-5. Specifically, the setting of 
the second selection section 27 is changed so that the registers 23aa to 23ac are 
connected to the multiplier 31a, the registers 23ba to 23bc are connected to the 
multiplier 31b, and the registers 23ca to 23cc are connected to the mxdtiplier 31c. Then, 
5 at step S12-6, the pixel data of the nine pixels stored in the third temporary storage 

section 23 is inputted to the arithmetic circuit 20, and the pixel data of the output pixel 
(l, 64) is generated. After the above-mentioned operation, the setting of the first 
selection section 26 is returned to the upper position. 

[0076] Fig. 30 shows the status of the pixel data stored in the first temporary storage 

1 0 section 21 and the second temporary storage section 22 immediately after the pixel 

data of the pixel (l, 66) is generated at time t+3, that is, immediately before step S12-1 
at time t+4. The pixel data of the pixels (3, 64) to (66, 64) is stored in the RAM 21a, the 
pixel data of the pixels (3, 65) to (66, 65) is stored in the RAM 21b, and the pixel data of 
the pixels (3, 66) to (66, 66) is stored in the RAM 21c. The pixel data of the six pixels (l, 

15 64) to (2, 66) is stored in the second temporary storage section 22. 

[0077] Fig. 31 shows the status of the pixel data stored in the first temporary storage 
section 21 and the second temporary storage section 22 immediately after step S12-1 at 
time t+4. At step S12-1, overwriting with the pixel data of the pixel (65, l) read from 
the memory 3 is carried out for the register 22ae in which the pixel data of the pixel (l, 

2 0 64) is stored. The speed, destination and direction of reading pixel data are determined 
according to Figs. 13 and 14. First, with respect to the condition 1, the previous 
writing point in the RAM 21c xmder writing is an address in which the pixel data of the 
pixel (66, 66) is stored, and the previous reading point is an address in which the pixel 
data of the pixel (3, 66) is stored. Therefore, the writing point is immediately before 

2 5 the reading point. Further, with respect to the condition 2, the pixel data of the pixels 
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(65, l) to (66, 3) required for generating the pixel data of the pixels in the next line (the 
first line) has not yet been stored in the second temporary storage section 22. 
Furthermore, with respect to the condition 3, the pixel (66, 66) read from the memory 3 
and written to the RAM 21c is not a pixel in the first three fines of the image block 13. 
5 Therefore, the loading method B is selected, whereby the pixel data of the pixel (65, l) 
is read fi:om the memory 3 and written to the register 22ae. 

[0078] Fig. 32 shows the status of the pixel data stored in the third temporary storage 
section 23 immediately after the pixel data of the output pixel (2, 64) is generated at 
step S12-6 of time t+4. At step S12-2 of time t+4, it is judged that the output pixel (l, 

1 0 64) generated previously does not correspond to the last pixel in a line of the image 

block 13. Further, at step S12-7, the pixel data in the third temporary storage section 
23 is shifted, and at step S12-8, the pixel data of the pixels (4, 64) to (4, 66) is 
respectively loaded from the RAMs 21a to 21c to the third temporary storage section 23. 
Then, at step S12-6, the pixel data of the nine pixels stored in the third temporary 

15 storage section 23 is inputted to the arithmetic circuit 20, and the pixel data of the 
output pixel (2, 64) is generated. 

[0079] Fig. 33 shows the status of the pixel data stored in the first temporary storage 
section 21 and the second temporary storage section 22 immediately before step S12-1 
of time t+5, and Fig. 34 shows the status of the pixel data stored in the first temporary 

2 0 storage section 21 and the second temporary storage section 22 immediately after step 
S12-1 of time t+5. The speed, destination and direction of reading pixel data firom the 
memory 3 are determined according to Figs. 13 and 14. With respect to the condition 1, 
the previoxis writing point in the RAM 21c xmder writing is an address in which the 
pixel data of the pixel (66, 66) is stored, and the previous reading point is an address in 

2 5 which the pixel data of the pixel (4, 66) is stored. Therefore, the writing point is not 
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immediately before the reading point. Fxirther, with respect to the condition 2, the 
pixel data of the pixels (65, l) to (66, 3) required for generating the pixel data of the 
pixels in the next line (the first line) has not yet been stored in the second temporary 
storage section 22. Furthermore, with respect to the condition 3, the pixel (65, 1) read 
5 fi-om the memory 3 and written to the register 22ae is a pixel in the first three lines of 
the image block 13. Therefore, the loading method H is selected, the pixel data of the 
pixel (65, 2) is read firom the memory 3 and written to the register 22be, and the pixel 
data of the pixel (67, l) is read from the memory 3 and written to the RAM 21a. 
[0080] Fig. 35 shows the status of the pixel data stored in the third temporary storage 

1 0 section 23 immediately after the the-output pixel (3, 64) is generated at step S12-6 of 
time t+5. At step S12-2 of time t+5, it is judged that the pixel (2, 64) generated 
previously does not correspond to the last pixel in a Une of the image block 13. Further, 
at step S12-7, the pixel data in the first temporary storage section 21 is shifted. At step 
S12-8, the pixel data of the pixels (5, 64) to (5, 66) is respectively read fi-om the RAMs 

15 21a to 21c and written to the third temporary storage section 23. Then, at step S12-6, 
the pixel data of the nine pixels stored in the third temporary storage section 23 is 
inputted to the arithmetic circuit 20, and the output pixel (3, 64) is generated. 
[0081] At times t+6 to t+9, as in the case of time t+5, the pixel data read fi'om the 
memory 3 is written to the first and second temporary storage sections 21 and 22 in the 

2 0 vertical direction (as indicated by the arrow "C" in Fig. 10). Fvirther, at time t+6 to t+9, 
four output pixels (4, 67) to (7, 64) are generated. As shown in Fig. 36, immediately 
before step S12-1 at time t+10, the pixel data of the six pixels (65, l) to (66, 3) has been 
stored in the second temporary storage section 22. Furthermore, the pixel data of the 
pixels (67, 1) and (68, l) is stored in the RAM 21a, the pixel data of the pixels (67, 2) 

25 and (68, 2) is stored in the RAM 21b, and the pixel data of the pixel (67, 3) is stored in 
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the RAM 21c. 

[0082] At step S12-1 of time t+10, the loading method G is selected according to Figs. 
13 and 14, and the pixels (68, 3) and (69, 1) read from the memory 3 are stored in the 
RAM 21a as shown in Fig. 37. Since the pixel (7, 64) generated previously is not the 
5 last pixel in a line at S12-2, the data in the first temporary storage section 21 is shifted 
at step S12-7. At step S12-8, the pixel data of the pixels (10, 64) to (10, 66) is 
respectively read from the RAMs 21a to 21c and stored in the registers 23aa, 23ba and 
23ca. In the end, at step S12-6, the arithmetic circuit 20 generates the output pixel (10, 
64) on the basis of the pixel data of the nine pixels (8, 64) to (10, 66) stored in the third 

1 0 temporary storage section 23. 

[0083] In the above-mentioned embodiment, the present invention is explained by 
taking an example wherein the pixel data of an output pixel is generated from the pixel 
data of nine adjacent pixels, that is, three adjacent pixels in the horizontal direction 
and three adjacent pixels in the vertical direction (see Figs. 8A to 8C). In case that "M" 

15 is an integer of two or more, "N" is an integer of two or more, and the arithmetic circuit 
generates the output pixel from the pixel data of M pixels in the horizontal direction 
and N pixels in the vertical direction, the first temporary storage section needs to 
readably store the pixel data of only the Mth pixel to the last pixel in each Une, and the 
second temporary storage section needs to readably store the pixel data of only the 

2 0 head pixel to the (M"l)th pixel in each Une. In addition, in this case, with respect to 
the pixels belonging to the first to the Nth lines of one image block, the pixel data of 
the pixels should only be loaded from the memory to the first and second temporary 
storage sections in the vertical direction as indicated by the arrow "C" in Fig. 10. 
[0084] The present invention is not Umited to the above-mentioned embodiments, but 

2 5 can be modified variously. For example, the present invention can be appUed to the 



resolution conversion circuit 6 in the image processing system 1 shown in Fig. 1. In 
addition, the present invention can also be applied to the image processing systems of 
apparatuses other than digital cameras, such as digital video cameras. Furthermore, 
the present invention can also be appUed to a case wherein one frame of image is 
processed without being divided into image blocks. 

[0085] Although explained completely referring to the accompanying drawings, the 
present invention can be changed and modified variously by those skilled in the art. 
Hence, it is understood that such changes and modifications are included in the 
present invention without departing from the spirit and scope of the present invention. 



ABSTRACT 

An image processing apparatus hae -having an arithmetic circuit 30-for generating 
pixel data of output pixels by arithmetically processing the pixel data of M horizontally 
adjoining pixels and N vertically adjoining pixels. A first temporary storage section-8i 
readably stores the pixel data of the pixels ranging from the Mth pixel to the last pixel 
in each horizontal pixel Une of an image. A second temporary storage section-33 
readably stores the pixel data of the pixels ranging from the head pixel to the (M-l)th 
pixel in each horizontal pixel line of the image. A third temporary storage section-^ 
delays the pixel data stored in the first temporary storage section-2i, receives the pixel 
data from the second temporary storage section-3S, and simultaneously outputs pixel 
data of the M horizontally adjoining pixels and the N vertically adjoining pixels. 



